Automatic Hybrid Approach for Lip POI Localization: Application for Lip-reading System

Automatic Lip-reading system is one of the different assistive technologies for hearing impaired or elderly people. We can imagine, for example, a dependent person ordering a machine with an easy lip movement or by a simple visemes (visual phoneme) pronunciation. The need for an automatic lip-reading system is ever increasing. The lip-reading system is decomposed in three subsystems, first we have the lip localization system, which localizes the lips in the digital input, then there is the feature extracting system, and finally we have the classification system, which maps feature vectors to visemes. The major difficulty of the lip-reading system is the extraction of the visual speech descriptors. In fact, to ensure this task it is necessary to carry out an automatic localization and tracking of the labial gestures. We present in this paper a new automatic approach for lip POI localization on a speaker’s face based on the color information of mouth and a geometrical model of lips. This hybrid solution makes our method more tolerant to noise and artefacts in the image. The algorithm works in natural condition and experiments revealed that our lip POI localization approach for lip-reading purpose is possible using the proposed approach.